PCA-Enhanced Stochastic Optimization Methods
نویسندگان
چکیده
In this paper, we propose to enhance particle-based stochastic optimization methods (SO) by using Principal Component Analysis (PCA) to build an approximation of the cost function in a neighborhood of particles during optimization. Then we use it to shift the samples in the direction of maximum cost change. We provide theoretical basis and experimental results showing that such enhancement improves the performance of existing SO methods significantly. In particular, we demonstrate the usefulness of our method when combined with standard Random Sampling, Simulated Annealing and Particle Filter.
منابع مشابه
Online Robust PCA via Stochastic Optimization
Robust PCA methods are typically based on batch optimization and have to load all the samples into memory during optimization. This prevents them from efficiently processing big data. In this paper, we develop an Online Robust PCA (OR-PCA) that processes one sample per time instance and hence its memory cost is independent of the number of samples, significantly enhancing the computation and st...
متن کاملCOMPUTER CODES FOR COLLIDING BODIES OPTIMIZATION AND ITS ENHANCED VERSION
Colliding bodies optimization (CBO) is a new population-based stochastic optimization algorithm based on the governing laws of one dimensional collision between two bodies from the physics. Each agent is modeled as a body with a specified mass and velocity. A collision occurs between pairs of objects to find the global or near-global solutions. Enhanced colliding bodies optimization (ECBO) uses...
متن کاملStochastic Optimization of PCA with Capped MSG
We study PCA as a stochastic optimization problem and propose a novel stochastic approximation algorithm which we refer to as “Matrix Stochastic Gradient” (MSG), as well as a practical variant, Capped MSG. We study the method both theoretically and empirically.
متن کاملA New Stochastic Optimization Algorithm based on a Particle Collision Metaheuristic
In many complex optimization problems, the so-called stochastic optimization methods as genetic algorithms and simulated annealing have been applied successfully. However, these methods have drawbacks: simulated annealing is too sensitive to the choice of parameters and the canonical genetic algorithm is liable to suboptimal convergence. With the purpose of overcoming these drawbacks, a new sto...
متن کاملDROP: Dimensionality Reduction Optimization for Time Series
Dimensionality reduction is a critical step in analytics pipelines for high-volume, high-dimensional time series. Principal Component Analysis (PCA) is frequently the method of choice for many applications, yet is often prohibitively expensive for large datasets. Many theoretical means of accelerating PCA via sampling have recently been proposed, but these techniques typically treat PCA as a re...
متن کامل